Skip to content

[DO NOT MERGE] Sch 1990 remove registry index#3610

Draft
emmalowe wants to merge 5 commits intomainfrom
SCH-1990-remove-registry_index
Draft

[DO NOT MERGE] Sch 1990 remove registry index#3610
emmalowe wants to merge 5 commits intomainfrom
SCH-1990-remove-registry_index

Conversation

@emmalowe
Copy link
Copy Markdown
Contributor

@emmalowe emmalowe commented Apr 30, 2026

In elasticsearch.yml there is a mapping of a "registry_index" to the government index. This reference is used in code that looks up linked documents to supported expanded links for four fields that could exist on an elasticsearch document:

  • policy areas
  • document_series
  • document_collections
  • world_locations

The policy_areas and document_series fields have been deprecated, meaning we no longer need the link expansion and so it's being removed. For the other two, we're switching to look up additional information from the govuk index instead of the government index. We're then deleting the reference to the registry_index, in preparation of deprecating the government index.

Jira ticket: https://gov-uk.atlassian.net/browse/SCH-1990

Migration of document_collections isn't working because documents with this format don't have the slug field populated. This needs sorting before merging.

The "document_series" field has been renamed to
"document_collections" [1]. There are no documents
in elasticsearch with this field populated, so we
no longer need the registry that expands the field.

[1] alphagov/whitehall#915
Registries are used to expand fields on elasticsearch
documents before they are presented to search API.
"document_collection" is a migrated format, so the field
"document_collections" should be expanded using content from
the govuk index, instead of the government index.

See PR where "document_collections" field was reinstated
for additional context:
#3215
The "policy_areas" field has been deprecated and is no longer
being indexed into elasticsearch [1]. There are no documents
in elasticsearch with this field populated in the govuk index,
so we no longer need the registry that expands the field.

[1] alphagov/whitehall#5666
Registries are used (in part) to expand fields on elasticsearch
documents before they are presented to search API. This works by
mapping the slug stored in the field to documents of the relevant
format, and taking additional information from the mapped document.

The "world_location" format is no longer being used: there
are no documents with this format stored in elasticsearch.
However, the field "world_locations" is used in many places
(for example as a filter on many finders) and the registry field
expansion changes the format of the field [1][2] from the raw
elasticsearch document from something that looks like:

"world_locations" : ["bulgaria"],

to something that looks like:

"world_locations": [{"slug": "bulgaria"}],

We don't yet understand the consequences of changing the formatting
on this field, so want to preserve it. As a quick way to keep this
formatting change, but remove references to the registry_index (as we
prepare to deprecate the government index), we can use the govuk
index to 'expand' the field. Even though it adds no additional
information (as before), this will preserve the formatting change.

[1] https://github.com/alphagov/search-api/blob/main/lib/search/presenters/entity_expander.rb#L75
[2] https://github.com/alphagov/search-api/blob/main/lib/search/presenters/field_presenter.rb#L33
Now all registries pull information from the govuk
index, we can remove the registry_index that points
to the government index.
@emmalowe emmalowe force-pushed the SCH-1990-remove-registry_index branch from 190cd6b to c7d5188 Compare April 30, 2026 17:39
@emmalowe emmalowe changed the title Sch 1990 remove registry index [DO NOT MERGE] Sch 1990 remove registry index May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant